I'd love to know if there is already a standard way to expand a token stream. I couldn't find one, so I did some refactoring here.

This is the relevant file for rustfmt: the parse_cfg_select function returns a CfgSelectBranches that keeps (as far as I can tell) all of the structure that is required to format cfg_select.

A cfg_select looks like this

fn print() {
    println!(cfg_select! {
        unix => { "unix" }
        _ => { "not unix" }
    });
}

where the right-hand side of the arrow is a TokenTree. That token tree is expanded based on the context: if the macro is in expression position, it'll be parsed as an expression, similarly for statements, items and any other position where a macro can occur.

So the formatter heeds to handle this TokenTree (really a TokenStream after we strip of the outer braces) somehow. I didn't see immediately how to do that given the existing APIs, but I assume it's possible because macros-by-example would need the same thing.

#[derive(Default)]
pub struct CfgSelectBranches {
    pub reachable: Vec<(MetaItemInner, TokenStream, Span)>,
    pub wildcard: Option<(Token, TokenStream, Span)>,
    pub unreachable: Vec<(CfgSelectRule, TokenStream, Span)>,
}

Question, what's the difference between reachable, wildcard, and unreachable?

Also, will the TokenTree on the right-hand side of the arrow always be valid rust?

Question, what's the difference between reachable, wildcard, and unreachable?

For formatting there is really no difference. The goal of this structure is to be able to emit warnings for unreachable branches.

The cfg rules on the left-hand side are evaluated from top to bottom, and the first one that evaluates to true is picked. The wildcard _ always evaluates to true, so any branches that follow it are unreachable.

Also, will the TokenTree on the right-hand side of the arrow always be valid rust?

I don't think so. It is like the right-hand side of a macro_rules! macro rule.

Can you provide examples of more complicated RHS that aren't valid rust?

Something like this

error: macro expansion ignores `+` and any tokens following
  --> /home/folkertdev/rust/rust/tests/ui/macros/cfg_select.rs:30:12
   |
LL | / cfg_select! {
LL | |     _ => { + + + }
   | |            ^
LL | | }
   | |_- caused by the macro expansion here
   |
   = note: the usage of `cfg_select!` is likely invalid in item context

the rhs is a valid token tree, but when expanded it's not valid rust code.

rustfmt would definitely fail to handle that. Is there a more realistic example you can think of?

In the example above things fail to compile, but is there a case where the RHS is composed of valid tokens that don't parse as valid rust, but still get expanded to valid rust?

Technically, rustfmt doesn't care what the tokens get expanded to as long as the tokens themselves can be parsed as valid rust, otherwise there's no hope to format them. Would you say that the typical case is going to be a RHS that parses as valid rust?

Yes, in valid programs the rhs will generally be valid rust. The only tricky thing is that it's unclear what kind (could be a sequence of items, or an expression, or a statement).

For now I think it should be enough to call format_snippet. That's how we currently format branches in macro_rules macros.